Lingua Cultura, 16(2), December 2022, 155-161

DOI: 10.21512/lc.v16i2.8315

P-ISSN: 1978-8118 E-ISSN: 2460-710X

GRAMMARLY AS AUTOMATED WRITING EVALUATION: ITS EFFECTIVENESS FROM EFL STUDENTS’ PERCEPTIONS

Utami Dewi*

Department of English Education, Faculty of Tarbiyah Science and Teacher Training, Universitas Islam Negeri Sumatera Utara

Jl. Wiliem Iskandar Pasar V, Medan Estate, Sumatera Utara 20731, Indonesia utamidewi@uinsu.ac.id

Received: 16th March 2022/Revised: 21st October 2022/Accepted: 21st October 2022

How to Cite: Dewi, U. (2022). Grammarly as automated writing evaluation:

Its effectiveness from EFL students’ perceptions. Lingua Cultura, 16(2), 155-161. https://doi.org/10.21512/lc.v16i2.8315

ABSTRACT

The research aimed to describe EFL students’perceptions of the effectiveness of Grammarly as an Automated Writing Evaluation (AWE). Nowadays, the Grammarly application is used to give AWE in many EFL writing classes. The respondents of the research were 75 students in an EFL writing class. The questionnaires and interviews were used to explore the students’ perceptions of using Grammarly in editing their drafts. The data from the questionnaires were calculated by percentages, and the data from the interview were analyzed by content analysis. The results show that the students have positive perceptions of using the Grammarly application as AWE. They state many advantages of Grammarly, such as Grammarly helps to identify errors in grammar, punctuation, spelling, and word choices in writing; helps to minimize errors in writing products; to enhance paraphrasing skills; to corrects the errors in writing anytime and anywhere; helps to improve students’ vocabulary; and Grammarly is easier to find errors in writing. Besides the advantages, the students state the drawback of Grammarly as AWE, such as Grammarly for free version has limited features to check errors; Grammarly for the premium version is expensive for students; and Grammarly cannot be accessed offline. The results of the questionnaires and interview represent that the Grammarly application has positive impacts on EFL students as an automated writing evaluation.

Keywords: grammarly, Automated Writing Evaluation, EFL writing class

INTRODUCTION

Amidst the COVID-19 era, the teaching- learning process cannot be organized face to face but through an online meeting. Because of this, the utilization of technology is unavoidable and necessary for students of English language learning. It makes Mobile-assisted Language Learning (MALL) used in the teaching-learning process. MALL is a language learning through technology. The use of MALL is also increasing as a consequence of online meetings. The students and teachers cannot meet face to face in the classroom, so they use some applications to improve their ability to learn English. MALL is defined as an approach to increase language learning by handheld mobile devices (Fithriani et al., 2019).

Technology developments make many computerized programs that support language learners to develop better. The development ofAWE (Automated Writing Evaluation) software has enabled L2 (Second Language) learners to receive feedback on language and content in addition to automated scores (Bailey & Lee, 2020). The usage of many automatic Writing assessments programs nowadays has emerged as a new trend in many EFL (English as a Foreign Language) writing lessons. Many researchers have stated that the software suggests a positive effect on the quality of students’ second drafts and subsequent submissions. Post-treatment questionnaires, interviews, and focus groups show a positive effect on the student’s attitudes toward feedback (El-Ebyary & Windeatt, 2010).

Writing is a complex skill that involves

expressing ideas in written language. In addition, writing is a group of letters or symbols written or marked on a surface to communicate an idea to others (Collin in Hakiki, 2021). Writing is such a complex activity that requires a lot of cognitive and linguistic abilities; consequently, EFL learners commit errors when completing a particular essay (Faller, 2018). EFL students have claimed that writing is difficult to learn because they lack vocabulary and grammar when writing a piece of writing (Hakiki, 2021). The development of technologies proposes many applications which assist students in solving their problems in writing. Many programs support or assist language learning. The use of MALL is evolving nowadays.

In the research, MALL also is used in EFL writing classes. In the writing process, EFL students struggle to produce the written language. Besides the students expressing their ideas in forms of written language, they also need to pay attention to vocabulary, grammar, punctuation, etc. Grammar, punctuation, capitalization, and spelling become vital elements in writing. Good grammar in writing helps writers deliver the message to the readers (Perdana & Farida, 2019). Most EFL students have the problems in writing, such as a lack of grammar and limited vocabulary. Undergraduate L2 students often struggle with paraphrasing, summarizing, synthesizing, appropriate attribution, and referencing, which may reduce motivation in the L2 writing class (Lazic et al., 2020).

One of the products is Grammarly as an AWE program, which evaluates writing automatically using an online system. Grammarly has two types: a free version and a premium version. Grammarly for free version has limited features which provide general aspects such as grammar, punctuation, and word spelling. Grammarly for the premium version has more completed features, such as plagiarism detection, vocabulary suggestions, and contextual spelling. In addition, the Grammarly premium version also gives more grammar errors and checks an additional 150 grammar points (Pratama, 2020). Figure 1 shows the differences between free and premium Grammarly.

Grammarly helps students edit their writing by checking potential writing issues with grammar, spelling, punctuation, and vocabulary suggestions. Grammarly is a simple tool that can help students and academies deal with their writing by checking spelling, grammar, and punctuation errors. It also provides comprehensive and beneficial feedback, such as corrections and suggestions to make the writing clearer, more precise, more effective, more readable, mistake-free, and impactful, with a high rate of accuracy and evaluation speed (Fahmi & Cahyono, 2021). Grammarly is one of the AWE programs used as computer-generated feedback.

Amidst the Covid-19 era, EFL writing classes are not organized face to face; thus, the lecturer cannot give the students corrective feedback or evaluation directly in the process of revising and editing essays.

In the editing process, the lecturer asks the students to use a tool to improve the quality of their writing. In the EFL writing class, the lecturer encourages the students to use technology in editing their essays. Technology development increases the uptake of software that can automatically analyze students’ writing (Hockly, 2019). In this research, the MALL assists students in improving their writing ability. It gives solutions for the students’ problems in writing, such as the Grammarly application used for Automated Writing Evaluation and feedback.

Figure 1 The Features of Grammarly in Free and Premium Version

In EFL writing class, the students use the free version of the Grammarly application autonomously to edit their essays. Grammarly is used as automated writing evaluation because the function of Grammarly is to give corrective feedback on students’ writing (Bailey & Lee, 2020). Before the students submit the essays to the lectures, they can use the Grammarly application to automatically check their corrective feedback from the tool. Grammarly is an application that can help to learn English in particular writing, such as Grammarly is a writing assistant, which means this application proofreads and edits documents for grammar and spelling (Fahmi & Rachmijati, 2021). The effectiveness of AWE software is a topic of some controversy among both researchers and classroom practitioners, with advocates viewing AWE as an excellent tool for improving students’ writing (Hockly, 2019).

Some studies have investigated the implementation of Grammarly and the benefit of Grammarly in writing classes. Grammarly is beneficial for EFL students as the assistant helps the teachers to guide the students on writing activities (Mubarok & Syafi, 2020). The presence of Grammarly is one of the online grammar checkers as the impact of technology development and the use of this software affects the

increase of students’ performance in writing (Fitria, 2021). Grammarly has various advantages, including grammar checker, proofreading, and plagiarism check. The Grammarly application can provide input recommendations on correct spelling and correct errors in word usage (Fahmi & Rachmijati, 2021; Hakiki, 2021). In addition, Grammarly is one of the convenient tools that encourage students to edit their writing by checking its spelling, grammar, and punctuation errors. Other researchers have revealed the positive effects of Grammarly on improving students’ writing, confirming the benefits of free Automated Writing Evaluation tools (Parra & Calero, 2019).

Previous studies have stated the effectiveness of the Grammarly application as AWE improves the students’ writing products in the EFL classroom. Those studies do not explore the students’ perceptions of using Grammarly in EFL writing classrooms. In accordance with the research findings, the research explores further the EFL students’ perceptions of using Grammarly as AWE in their writing classroom. Firstly, the research describes whether the students have positive or negative perceptions of using Grammarly. Secondly, students’ perceptions of using Grammarly and its effectiveness in editing their essays. Thirdly, the advantages and drawbacks of Grammarly are explored based on students’ experiences in using Grammarly in EFL writing classes. The findings of the research describe the students’ perceptions of using the Grammarly application as AWE in EFL writing class.

METHODS

The qualitative research describes students’ perceptions of effectiveness of the Grammarly application as an Automated Writing Evaluation (AWE). The data are described qualitatively based on the students’ experience using the Grammarly application. The students are given questionnaires and interviews. Purposive sampling is used to choose respondents to the research. The respondents are selected because the students use Grammarly in their EFL writing classroom in order to get data about their perceptions of its use. The respondents are 75 students who studied EFL writing classes. In EFL writing class, the students are given the assignment to write some essays, and they follow the writing process. In the editing process, the students are suggested to use the Grammarly application to edit their drafts. The students ask to use the free version of Grammarly. They use the Grammarly application six times in the process of editing their essay.

The instruments of data collection are questionnaires with a Likert scale and interviews. Firstly, 75 students are given questionnaires with five scales to investigate their perceptions on the use of Grammarly as AWE in their EFL writing class. The questionnaires consist of two sections and eleven statements that explore the intensity and effectiveness of Grammarly usage. Next, the interview explores the

advantages and weaknesses of Grammarly’s application as AWE. From the 75 students, the researcher has selected 15 to be asked two questions confirming their answers in the questionnaires. The 15 students are selected randomly and consisted of the students with positive or negative perceptions; therefore, they share their perceptions on using the Grammarly fee version in their writing class. The interview questions confirm the advantages and drawbacks of Grammarly based on students’ experience in EFL writing classes. The students with positive perceptions are asked about the advantages of Grammarly; otherwise, the students with negative perceptions are asked about the drawbacks of Grammarly.

The research data are analyzed using percentages and content analysis. Firstly, the data from questionnaires are analyzed by percentages. The analysis describes the percentages of the effectiveness of students’ perceptions of Grammarly as AWE in their EFL writing class. Secondly, the data from the interview are presented by theme categories. The results show some of the advantages and drawbacks of Grammarly as AWE in the EFL writing class.

RESULTS AND DISCUSSIONS

The findings are found by analyzing the questionnaires and interviews given to the students. Firstly, the findings of this research represent the intensity of using the Grammarly application in their writing class and the students’ intensity in reading the comments when they use the Grammarly application. The questionnaires are about the student’s perceptions of using the Grammarly application in their writing process. The data of the questionnaires are analyzed by calculating the percentages of the total number of students that answer of never, occasionally, sometimes, often, and always that can be seen in Figure 2.

Figure 2 The Use of Grammarly

Q1: I used Grammarly in editing essay.

The results of the first questionnaires in the first section present that none of the students ever used Grammarly in editing, 10,7% of the students occasionally use Grammarly in editing, 53,3% sometimes use Grammarly in editing, and 20% of the students often use Grammarly in editing. Based on the result, most of the students ever use Grammarly to edit their essays. The percentages point out that all students have ever used the Grammarly application to check their errors in writing. Figure 3 is one example of using Grammarly, where it is NK’s screenshot of using Grammarly as AWE.

Q2: When using Grammarly, I read an extensive commentary.

In addition, the results of the second questionnaire

of the first section describe when using Grammarly. 5,3

% of the students never read the extended explanation of errors, 4% of the students occasionally read the extensive commentary, 16% of the students sometimes read the extensive commentary, 38,7% of the students often read the extended explanation of errors, and 36% always read the extensive comment of mistakes. The results mean that most students read the comprehensive analysis of errors when editing their writing using the

Grammarly application. They learn how to edit their mistakes from the Grammarly analysis.

Secondly, the findings of the research describe the students’ perceptions of using Grammarly as MALL and AWE in editing their essays. Table 1 describes the results of the questionnaires of the students’ perceptions of Grammarly as MALL and AWE. The questionnaires are analyzed by calculating the total number of students who strongly disagree, disagree, neutral, agree, and strongly agree. After getting the total numbers, the results are calculated by percentages in each item of the questionnaires. In Table 1, it is written the categories in numbers 1-5 that represented strongly disagree (1), disagree (2), neutral (3), agree (4), and strongly agree (5).

Those percentages reflect that most of the students have positive perceptions of the use of Grammarly as MALL. Other research also describes that the students give positive responses to the use of Grammarly (Fahmi & Cahyono, 2021). Most of the students are ‘strongly agree’ and ‘agree’ with the statements in the questionnaires. The questionnaires’ findings represent the students’ positive perceptions of Grammarly as AWE. They consider Grammarly is an effective tool for automated writing evaluation. Grammarly helps students identify errors simpler and

Figure 3 The Example of Student’s Screenshot in Using Grammarly

Table 2 Students’ Perceptions of Grammarly as MALL and AWE

quicker; minimize mistakes in grammar, punctuation, and word choices; enhance paraphrasing skills; to improve summarizing skills. In other research, the students review Grammarly as an effective tool because Grammarly can be easy to use, accessed quickly, and helpful in checking grammar errors. Grammarly is one of the online grammar checkers used in assessing EFL writing classes and proofreading by checking mistakes such as grammar (Fitria, 2021). Grammarly is accurate, precise, detailed, and can be accessed anytime and anywhere. Grammarly is one of the possible solutions that help students correct their language (Mubarok & Syafi, 2020). Based on the finding, the students consider Grammarly as one of the applications that assisted them in editing their draft.

Furthermore, the research also uses interviews to explore more students’ perceptions of the effectiveness of Grammarly as AWE. A structured interview is

given to the students to explore the advantages and drawbacks of Grammarly application as AWE. The first interview is, “What are the advantages of using Grammarly?” The transcription of the interview is analyzed by content analysis in Table 3.

Table 3 represents the transcription of the interview that analyzes by content analysis. The interview transcription is classified into six theme categories as the advantages of Grammarly as AWE. The six advantages of Grammarly as an automated writing evaluation are that it helps to identify errors in grammar, punctuation, spelling, and word choices in writing; to minimize errors in writing products; to enhance paraphrasing skills; to correct errors in writing anytime and anywhere; to improve students’ vocabulary; and to find errors in writing easier.

Based on the students’ experience in using Grammarly as AWE, they have stated that Grammarly

Table 3 Advantages of Grammarly as AWE

Table 4 The Drawback of Grammarly as AWE

facilitates to perceive of mistakes in grammar, punctuation, spelling, and phrase alternatives in writing; Grammarly enables to minimize errors in writing products; Grammarly helps the students learn how to paraphrasing; Grammarly can be accessed flexibly in time and place; Grammarly helps to improve the students’ vocabulary; and Grammarly identify the errors easier. These advantages of Grammarly are described that the students consider Grammarly as an online supplementary instructional tool to evaluate the student’s writing products. As in previous research, participants have positive perceptions of Grammarly and found it helpful in addressing shortcomings in their grammar knowledge, word usage, style, and writing mechanics. Other researchers have also stated the positive effect of Grammarly. They describe the positive impact of Grammarly software on EFL writing products. Grammarly software in EFL writing makes a positive contribution to reducing errors made by the students in terms of vocabulary usage (diction), language use (grammar), and mechanics (spelling and punctuation) (Ghufron & Rosyida, 2018; Hockly, 2019). In addition, Grammarly gives feedback and a guide to find the correct one (Fahmi & Rachmijati, 2021). These findings highlight the potential of predictive text and real-time corrective feedback as a way to support L2 writing, particularly among novice writers who may struggle to write effectively in the L2 (Dizon & Gayed, 2021).

Besides the advantages of Grammarly, the research also examines the drawbacks of Grammarly as AWE (Table 4). The second question is used to ask about the drawbacks of the Grammarly application for AWE. The question is, “What is the drawback of Grammarly?”

The analysis of the students’ interviews shows that some students stated that Grammarly has no weaknesses. Otherwise, some students have stated the drawbacks of Grammarly, such as Grammarly

for the free version having limited features to check errors, Grammarly for the premium version being expensive for students, and Grammarly cannot be accessed offline. Grammarly-free user has limitation because Grammarly is used effectively to cast off basic errors, grammar, punctuation, and conciseness, while Grammarly premium includes a variety of extra features consisting of readability sentence rewrites, tone adjustments, plagiarism detection, word choice, formality stage, fluency, and extra superior pointers.

Though there are many strengths of Grammarly software in the process of writing feedback, there are also some weaknesses of Grammarly in giving automated feedback-based EFL students’ experiences. Other researchers have also found some weaknesses in Grammarly. The students also admit that they face several weaknesses in utilizing Grammarly in comparing their instructional writing, consisting of deceptive comments, over-checking on reference lists, and inability to check the context and writing content (Nova, 2018). The findings show that Grammarly free has many advantages and few drawbacks. Grammarly suggests corrections, gives brief explanations about writing errors, helps them form sentences, and improves their overall writing performance. Grammarly free version can be considered as AWE for students in EFL writing class.

CONCLUSIONS

In the research, most of the students have positive perceptions of the use of Grammarly in their EFL writing classroom though they only use the free version of Grammarly. They find many advantages, such as it helps them to check the errors in grammar, punctuation, spelling, and word choice, helps them to minimize their mistakes in writing, and helps them to improve their vocabulary and writing ability. In

addition, students canaccess Grammarly easily, anytime and everywhere. By those many advantages found by students, it means that Grammarly is an effective application for Automated Writing Evaluation (AWE). Otherwise, some students perceive that Grammarly for the free version as AWE has some drawbacks, such as limited features to check errors, so if the students want to get further features in evaluating their writing, they need to use Grammarly for the premium version, which is expensive for students. Another weakness is that Grammarly should be used online.

Further research can show the effectiveness of Grammarly compared with other applications such as AWE or teachers’ feedback. Based on the research finding, it is suggested that the Grammarly for free version can be used as an online application that assists students in assessing their writing automatically and independently. In the limited time, the researcher does not compare the online feedback and direct feedback. Furthermore, the other researchers can explore or compare the automated writing evaluation and peer feedback in the process of revising in EFL writing classrooms.

REFERENCES

Bailey, D., & Lee, A. R. (2020b). An exploratory study of Grammarly in the language learning context: An analysis of test-based, textbook-based, and Facebook corpora. TESOL International Journal, 15(2), 4-17.

Dizon, G., & Gayed, J. M. (2021). Examining the impact Of Grammarly on the quality of mobile L2 writing. JALT CALL Journal, 17(2), 74-92. https://doi. org/10.29140/jaltcall.v17n2.336.

El-Ebyary, K., & Windeatt, S. (2010). The impact of computer-based feedback on students’ written work. International Journal English Studies, 10(2), 121-

142. http://dx.doi.org/10.6018/ijes.10.2.119231. Fahmi, M. A., & Cahyono, B. Y. (2021). EFL students’

perception on the use of Grammarly and teacher feedback. JEES (Journal of English Educators Society), 6(1), 18-25. https://doi.org/10.21070/jees. v6i1.849.

Fahmi, S., & Rachmijati, C. (2021). Improving students’ writing skill using Grammaly application for second grade in senior high school. PROJECT (Professional Journal of English Education), 4(1), 69-74. http:// dx.doi.org/10.22460/project.v4i1.p69-74.

Faller, J. M. V. (2018). Grammarly investigation into EFL writing issues involving Omani learners. International Journal of Language & Linguistics, 5(3), 165-174. https://doi.org/10.30845/Ijll. V5n3p16.

Fithriani, R., Dewi, U., Daulay, S. H., Salmiah, M., & Fransiska, W. (2019). Using Facebook in EFL writing class: Its effectiveness from students’ perspective. KnE Social Sciences, 634-645. https:// doi.org/10.18502/Kss.V3i19.4892.

Fitria, T. N. (2021). Grammarly as AI-powered English writing assistant: Students’ alternative for English writing. Metathesis Journal of English Language Literature and Teaching, 5(1), 65-78. http://dx.doi. org/10.31002/metathesis.v5i1.3519.

Ghufron, M. A., & Rosyida, F. (2018). The role of Grammarly in assessing English as a Foreign Language (EFL) writing. Lingua Cultura, 12(4), 395-403. https://doi. org/10.21512/lc.v12i4.4582.

Hakiki, G. N. R. (2021). EFL students’ perception of the use of Grammarly application in writing class. Eduvelop: Journal of English Education and Development, 4(2), 99-106. https://doi.org/10.31605/Eduvelop. V4i2.891.

Hockly, N. (2019a). Automated writing evaluation. ELT Journal, 73(1), 82-88. https://doi.org/10.1093/Elt/ Ccy044.

Lazic, D., Thompson, A., Pritchard, T., & Tsuji, S. (2020). Student preferences: Using Grammarly to help EFL writers with paraphrasing, summarizing, and synthesizing. In F. Karen-Margrete, L. Sanne;

B. Linda, T. Sylvie (Eds), CALL For Widening Participation: Short Papers from EUROCALL 2020 (pp. 183-189). https://doi.org/10.14705/ rpnet.2020.48.1186.

Mubarok, A., Syafi, H. (2020). Grammarly : An online EFL writing companion. ELTICS (English Language Teaching and English Linguistics) Journal, 5(2), 1-10. https://doi.org/10.31316/eltics.v5i2.912.

Nova, M. (2018). Utilizing Grammarly in evaluating academic writing: A narrative research on EFL students’ experience. Premise: Journal of English Education and Applied Linguistics, 7(1), 80-96. http://dx.doi.org/10.24127/pj.v7i1.1332.

Parra, L. G., & Calero, X. S. (2019). Automated writing evaluation tools in the improvement of the writing skill. International Journal Instruction, 12(2), 209-

226. https://doi.org/10.29333/Iji.2019.12214a.

Perdana, I., & Farida, M. (2019). Online grammar checkers and their use for EFL writing. Journal of English Teaching, Applied Linguistics and Literature (JETALL), 2(2), 67-76. http://dx.doi.org/10.20527/ jetall.v2i2.7332.

Pratama, Y. D. (2020). The investigation of using Grammarly as online. English Ideas: Journal of English Language Education, 1(1), 46-54.